Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Format and Lint to Makefile #545

Merged
merged 18 commits into from
Mar 18, 2020
Merged

Conversation

woop
Copy link
Member

@woop woop commented Mar 17, 2020

What this PR does / why we need it:

Refactors the root Make file so that we can format and lint for Java, Python, and Golang.

Which issue(s) this PR fixes:

This PR works towards fixing #526, but does not address all the necessary requirements to close it.

Does this PR introduce a user-facing change?:

None

@woop
Copy link
Member Author

woop commented Mar 17, 2020

I think we still need two or three things before we can close #526.

  1. A dedicated task that runs these checks and makes sure all failures are visible before exiting
  2. A means of pinning versions so that protoc (and its plugins) as well as the other formatters are always consistent. Alternatively we can also remove the auto generated code from the repository.

@woop
Copy link
Member Author

woop commented Mar 17, 2020

/test test-python-sdk

@woop
Copy link
Member Author

woop commented Mar 18, 2020

The latest commit also removes Python proto generated code.

@woop woop changed the title Add Format and Linting Add Format and Lint to Makefile Mar 18, 2020
pre-commit
flake8
black
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would propose to set the version for the linters like grpcio-tools, mypy, black, flake8 etc.
It would be awesome when we can be sure on same version when developing

Rest is LGTM :) The import structure and linter, format would be really helpful for anyone developing next

Copy link
Member Author

@woop woop Mar 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but we also dont want to be too restrictive. If we force everyone to conform to our versions then its also quite high friction. Only protoc was a problem, and with the addition of this PR we won't be version controlling proto generated Python code, so the problem kind of goes away.

I think we can pin the versions once it becomes a problem

Copy link
Contributor

@imjuanleonard imjuanleonard Mar 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "compatible" specifier might be a reasonable compromise, e.g. approximately expecting Semantic Versioning if a tool doesn't break behavior within a major version series:

flake8 ~= 3.0

sdk/python/feast/__init__.py Show resolved Hide resolved
@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: imjuanleonard, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@woop woop added the lgtm label Mar 18, 2020
@feast-ci-bot feast-ci-bot merged commit 3b15b14 into feast-dev:master Mar 18, 2020
cd ${ROOT_DIR}/sdk/go; gofmt -s -w *.go

lint-go:
cd ${ROOT_DIR}/sdk/go; go vet; golint *.go
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If my version of golint is still current, it requires a flag -set_exit_status in order to return an error status to the shell.

A formatting check should probably be included in lint-go as well, it is for the other languages. gofmt has no such feature at all to return an error exit status, the idiom seems to be to use test -z $(gofmt -l ./) when this is desired.

I'm not sure if we're actually wanting it to fail on CI yet, because of the desire to not fail lint checks one-at-a-time, but we probably want to be consistent about whether all the lint-* make targets exit nonzero when there's an issue (I haven't checked behavior of all others yet).

Probably lint-go will not fail CI for now even if this target is updated to exit nonzero, because .prow/scripts/test-golang-sdk.sh does not set -e.

@khorshuheng khorshuheng mentioned this pull request Apr 7, 2020
khorshuheng pushed a commit to khorshuheng/feast that referenced this pull request Apr 14, 2020
* Add Make commands for format, lint, flake8, spotless, isort, black, and refactor

* Add mypy test

* Add lint tests to CI

* Fix broken Python test

* Fix broken test for Python

* Add black to dependencies

* Remove Python Protos

* Add automatic local linting

* Update precommit names

* Add black exclusions

* Add tensorflow metadata proto generation

* Ignore tf meta directory

* Add build essentials to install make in CI

* Add exports back to __init__.py

* Add __all__ to export

* Add white space to export

* Add source to export

* Fix python export formatting
khorshuheng pushed a commit that referenced this pull request Apr 14, 2020
* Add Make commands for format, lint, flake8, spotless, isort, black, and refactor

* Add mypy test

* Add lint tests to CI

* Fix broken Python test

* Fix broken test for Python

* Add black to dependencies

* Remove Python Protos

* Add automatic local linting

* Update precommit names

* Add black exclusions

* Add tensorflow metadata proto generation

* Ignore tf meta directory

* Add build essentials to install make in CI

* Add exports back to __init__.py

* Add __all__ to export

* Add white space to export

* Add source to export

* Fix python export formatting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants